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. . According to A.A.Kirillov's orbit method there exists one to one correspon- 

^ ; dence between the irreducible representations of an arbitrary connected, simply 
^! connected Lie group and its coadjoint orbits. This correspondence between or- 

■ bits and representations makes possible to solve problems of the representation 
Q theory in terms of coadjoint orbits. The orbit method initiate many papers 

01 beginning from 1962. It turns out that the ideas of the orbit method are useful 
^ ! for the large classes of Lie groups (see P, S]), and also for some matrix groups, 

defined over finite field. 
P^^ ■ In our paper we obtain formula for multiplicities of certain representations 
^[ of unipotent groups over finite field in terms of coadjoint orbits (see theorem 
and corollaries). For reader's convenience we formulate and prove the maim 
statements of the orbit method over finite field (see [5]). 

Let K = ¥q he a. finite field of characteristic p having q = elements. Let 
O ! be a subalgebra of the Lie algebra ut(A^, K)^ consisting of all upper triangular 
matrices with zeros on the diagonal. Suppose that p is large enough to determine 
the exponential exp(a:) map on q. For instance, let p ^ N. Then the exponential 
map is a bijection of the Lie algebra g onto the subgroup G = exp(0) of the 



00 
OS 



^ • unitriangular group \JT(N, K). One can define the adjoint representation of 



the group G on g by the formula Adq{x) = gxg 



-1 



'. Denote by q* the conjugate space of q. One can define the coadjoint repre- 

■ -L-^: r -Li ^ ^* u,. r 1„ aj*\/™\ \/aj-1™\ 

Note that if is a stabilazer of A G 0*, then the subgroup G = exp(0^) is a 
stabilizer of A in G. One can calculate the number of elements of the orbit 
Q = Ad^(A) by the formula 

lOl — — 1^1 _ dimg-dimg-^ _ idimri f-i\ 

' '"iG'^riflT ~ ■ 

Definition 1. A subalgebra p of g is a polarization of A G 0*, if p is a maximal 
isotropic subspace for the skew symmetric bilinear form B\{x^ y) = A([a:, y\) on 
0. Recall that the subspace p is isotropic, if B\{x, y) = for any x,y ^ p. 
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Note that any polarization contains the stabilizer g'^, because p + g is an 
isotropic subspace. 

Proposition 2. Any linear form A on a nilpotent Lie algebra g has a polariza- 
tion. 

Proof. We shall prove using induction method for the dimension of the Lie 
algebra g. The statement if obvious for one dimensional Lie algebras, since in 
this case Lie algebra is a polarization. Assume that the statement is proved for 
all Lie algebras of dimension < dim(g). We are going to prove the statement 
for dim(g). 

If the dimension of a center 3 the Lie algebra g greater than one, then one 
can prove existence of polarization applying induction assumption for the factor 
algebra of g with respect to the ideal Ker(A|3). Similarly, for the case dim(3) = 
1, A|3 = 0. 

Let 3 = Kz and X{z) ^ 0. Consider the two dimensional ideal Ky + Kz, 
containing 3. There exists a character a of the Lie algebra g such that ady (?/) = 
a{u)z for any u E g. The kernel go of the character a is an ideal of codimension 
one in g. There exists an element x E g such that g = Kx + go and [x, y] = z. 

Denote by Aq the restriction of A on gQ. According the induction assumption 
Ao has a polarization po in Qo- Let us prove that po is also a polarization for A 
in g. Really, po is a subalgebra and a maximal isotropic subspace in go; we will 
show that po is a maximal isotropic subspace in g. Suppose that one can extent 
po to an isotropic subspace adding the element x + uq, where uq G go. Note 
that z, y belong to the stabihzer C po- Then = A([x + uq, y]) = A([x, y]) = 
X{z) 7^ 0. A contradiction. □ 

Proposition 3. Let p be a polarization of X E g*, P = exp(p), Q(A) be the 
coadjoint orbit of A, tt be the natural projection of g* onto p*, = 7r"V(A). 
Then 

1) dimp = i (dimg + dimg^); 

2) |L^| = vW)l; 

2) = AdpA, in particular C ^(A). 

Proof. The statement 1) follows from the formula of dimension of a maximal 
isotropic subspace for the skew symmetric bilinear form Bx{x^y). From 1) we 
obtain ^ ^ 

codimp = - (dimg — dimg^) = - dim 0(A). 

This implies the statement 2): 

|^A| _ ^codimp _ ^\div.n{X) ^ 
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Since A([a:, y\) = for any x, G p, we have adpA(?/) = for any y ^p. Then 
AdpA(?/) = \{y) for any G p. This is equivalent to 

Ad^A C L\ 

The equahty AdpA = is true, since this subsets have equal number of 
elements: 

|Ad^A| = 1^ = /^P-'l^fl' = g|dimO(A) ^ |^A|_ □ 

Fix a non trivial character : il' — > C*. We have 

E^at _ \ 1^ « = 0? (r,\ 
~ 1 0, a 7^ 



The equality (ED is easy to prove: the image of homomorphism is a subgroup 
of C*; if a 7^ 0, then this subgroup is nontrivial and coincides with the subgroup 
of all roots of some order m 7^ 1 of unity; the sum of all roots of order m 7^ 1 
of unity equals to zero. 

Restriction of A on its polarization p defines a character (one dimensional 
representation) ^ of the group P = exp(p) by the formula 

6(exp(x)) = e^(-). 

Consider the induced representation 

r^ = ind(^A,P,G). (3) 

Denote by Xx{9) ~ Tr T^{g) the character of representation T^. 
Theorem 4. 

' /ier!(A) 



Proof. Extent from P to G by the formula 

Un) = 

Formula (E]) implies 



^x{u) , if w G P, 
, if w ^ P. 



fi<EL^ ' ' peP 



Choose the system of representatives {gi : i = l,k} of the classes G/P. 
Using the well known formula for induced characters (see [6, chapter 6]), we 



obtain 

gi'ug^eP ' ' «=i,pe^ ' ' i=i,peP 



Finally, 



u(ln(u)) r-| 



Theorem 5. 

1) dimT^ = g5dimf^(A) _ 

2) The representation T'^ does not depend on the choice of polarization. 

3) The representation is irreducible. 

4) Representations and T^' are equivalent if and only if A and A' belong to 
the same coadjoint orbit. 

5) For any irreducible representation T of the group G there exists A G 0* such 
that the representation T is equivalent to T'^. 

Proof. The statement 1) follows from 



dimT^ = dim(ind(a, P, G)) = g^^dimp 

The statement 2) is a corollary of the theorem [H 

Let us show that the system of characters {xa}? where A is running through 
some system of representatives of the coadjoint orbits, is orthonormal. Let 
W be two coadjoint orbits and A, A' be representatives of this orbits. Then 

(XA,XA') = t;LX^Xa(w)xa'(w) = ■ ] ■ Y = 

1 M^"i^')Hu) 



\G\ ^\n\ ■ \n'\ 

Applying 

|G| , if r/ = 0, 
, if 



E 

we obtain that, if Q 7^ Q', then (xa, Xx') = 
In the case Q = W, we have got 



This proves 3) and 4). 



We shall use notation T for the class of equivalent representations T , where 
A G Q. To prove statement 5) we verify that the sum of squares of dimensions of 
irreducible representations {T^ : Q G Q*/G} equals to the number of elements 
of the group: 

^ {dimT"f= ^ (v^)'= ^ |fi| = |B*| = |G|. 

neg*/G n(EQ*/G Oer/C 

This proves 5).n 

Lemma 6. Let g be a nilpotent Lie algebra, Qo be a subalgebra of q of 
codimension one. Then Qq is an ideal of g. 

Proof. Suppose the contrary. Then [q,Qo] 7^ Qo] the exist the elements y G Qq, 
X ^ Qo such that [x^y] = ax mod 0o? a ^ 0. The subalgebra Qq is invariant 
with respect to ady^. Since g = kx ® Qq, the operator ady^ is not nilpotent in 
q/qo; this contradicts to assumption that the Lie algebra g is nilpotent. □ 

Lemma 7. Let q, Qq be as in lemma El tt is a projection q* — > Qq] Aq G Qq, 
u = Ad^^(Ao), 0^° = {xeQ: Ao([x,0o)] = 0}. 

1) Let the subalgebra 0^" belong to Qq. Then 
la) 7r^^(Ao) lie in the same Ad^-orbit 

lb) dimQ = dimcj + 2 (i.e. \Q\ = q'^\uj\) ■ 

2) Let the subalgebra 0^° do not lie in Qq. Then for any Ad^-orbit which 
has nonempty intersection with 7r^^(Ao), the projection tt establishes one to one 
correspondence between and cj; in particular, dimil = dimcj. 

Proof. 

1) Suppose that the subalgebra belongs to Qq. Since C Qq, the formula 
BQ{x,y) = Xo{[x,y]) defines a skew symmetric bilinear form on the Lie algebra 
Q. The kernel V of the bilinear form Bq coincides with 0^° and belongs to Qq. 
The kernel Vq of the restriction Bq on Qq coincides with 0q°. 

Let us prove that there exists a pair of elements u E q\qo and -u G Vq, such 
that Bq{u, v) = 1. Really, decompose go = ^o® ^o? where Lq is a subspace, with 
the property that the restriction of bilinear form Bq on Lq is nondegenerate. 
Choose an arbitrary element u' E q \ Qq and consider the linear form Bq{u', •) 
on Lq. There exists xq G Lq such that Bq{u', •) = Bq{xo, •) on Lq. The element 
u = u' — xq satisfies Bq{u., Lq) = 0. Since u ^ there exists v eVq such that 
Bq{u,v) = 1. 

By direct calculations, we verify that for any A G 7r~^(Ao) the following 
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equalities are valid 

I Ad*^p(t«)^(^) = Ku) + t, 

I Ad*^p(i^)A(?/) = X{y) for any 7/ G go- 

This implies the statement la). 
The orbit is a union 

n=\J 7r^\ut), (5) 
teK 

where cut = Adg^pj-^^^cj is a coadjoint orbit in Qq. Let us show that the orbits Ut 
are pairwise different. Really, if not, there exists t' ^ t" E K and q'q, Qq G Gq 
such that 



Then 



Ad*xp(i/^)AdJ/ Ao — Ad*xp(^„y)Ad*//Ao. 



Ad*xp(i^i)Ad*^Ao - Ao, 



where t = t' — t" E K* and go is an element of Gq. Then the stabilizer exp(g'^") 
does not belong to Go; this contradicts to assumption of the item 1). Using ([5]), 
we obtain \Q\ = q'^\uj\. Therefore dimil = dimoj + 2. This proves lb). 

Turn to proof of the item 2). Suppose that the subalgebra does not belong 
to Qq. For an arbitrary nonzero element x of \ Qq we have decomposition 
Q = Kx Qq. The group G is a semidirect product G = GqX, where X = 
{exp{tx) : t G K}. 

Let A G 7r"^(Ao). The equality 

X{[x, q]) = X{[x, Kx e Qq]) = Xo{[x, Qq]) = 

implies that x belongs to q'^. The subgroup X lies in the stabilizer of A. The 
orbit Q{X) coincides with Ad^^(A). Since the projection tt : q* ^ Qq is invariant 
with respect to Ad^^, the map tt project Q(A) onto u. 
It remains to show that 

f](A) n TT-^Ao) = {A} (6) 

for any A G 7r^(Ao). Suppose that A' = Ad* A and A, A' G 7r~^(Ao). Since 
G = GqX and X G G^, we verify that A' = ^d*^-^ some qq G Gq. As 
A, A' G 7r^^(Ao), the element qq lies in stabilizer Gq'. Then qq = exp(yo) for 
some uq G Qq°. We obtain 

X\x) = A(Ad*^p(_y^)x) = X{x) - X{ady^x) + ^ — ^Ao(adJ^x). (7) 

k^2 
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Since x G Q^, we have A(adyQa:) = 0. As yo G 0q°, we have Ao(ady^x) = for any 
k ^ 2. Substituting into ([7]), we obtain \'{x) = X{x). Using A, A' G 7r"^(Ao), we 
conclude A = A'. □ 

Lemma 8. For any subalgebra f) of a nilpotent Lie algebra q there exists a 
chain of subalgebras Q = Qo D Qi D . . ■ D Qk = ^ such that Qi+i is an ideal of 
codimension one in gi for any 1 ^ i ^ k — 1. 

Proof. We use the induction method for dimg. For dimg = 1 the statement 
is obvious. Assume that the statement is true for dimg = n — 1; let us prove 
it for n. The nilpotent Lie algebra g has a nonzero central element z. Con- 
sider projection : g — > g = g/Kz. The image () is a subalgebra in g. As 
dimg < n, according to induction assumption, there exists a chain of subalge- 
bras = D 01 D • • • D 0/j = J), where g^ is an ideal of codimension one in 
g^j^i for any 1 ^ i ^ k — 1. Denote gi = If 2 G f), then g^ = (); this 

completes construction of the chain of subalgebras. If z ^ f), then 0a: = f) + Kz. 
It remains to put gk+\ equal to f). □ 

Theorem 9. Let G = exp(0) be an unipotent group over the finite field 
K, f) be a subalgebra of H = exp{^). Let (resp. uj) be a coad- 
joint orbit in g* (resp. f)*), and be corresponding irreducible rep- 

resentations of G and i7, tt be the natural projection g* onto f)*. Denote 
m{uj, = mult(r", ind(t^, G)) = mult(^^, res(r", H)). Then 

|7r-Hc^)nQ| 
y |a;| • \U\ 

Proof. Introduce notations 

P = \7r-\u) n Q|, Q = • 1^1, M = mult (T^, md{r, G)) . 

We shall prove that M = P/Q using the induction method with respect to 
codim((),g). If codim((),0) = 0, then Q = co and hence P = Q = 

M = 1; this proves the equality M = P/Q. 

Assume that the equality is proved for codim(f),0) < k; let us prove for 
codim([), g) = k. The lemma E] implies that there exists a subalgebra g\ obeying 
the conditions D 0i D f), codim([),0i) = 1. Choose Aq G uj. The natural 
projections tt : 0* — > [)*, tti : — > g^, ttq : 0i — > i)* satisfy tt = ttotti. For 
01° = {3: G 01 : Ao[a:, ()] = 0} only two cases are possible: g\^ C f), or 0^° (/: [). 
1) Case 0^° C f). Following lemma [71 7rQ"^(a;) belongs to the same coadjoint 
orbit ill C 01 of the group Gi = exp(0i). Therefore, dimQi = dimcj + 2 and 

\Vti\=q^\uj\. (8) 
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A polarization pQ for Aq in f) is also a polarization for any Ai G ttq ^ in ^i. 
Really, po is an isotropic subspace in Qi and 

codim(po, Qi) = codim(po, + ! = -(dimw + 2) = - dim Qi. 

The induced representation md{t^, Gi) is irreducible and coincides with T^^, 

md{T^\G) = md{r,G). (9) 
According to the induction assumption, 



mult(T",ind(T".,G))=^4^£i£B 



(10) 



Applying the formula (Ej), we obtain 



Using dHD, we conclude 



M = , = ^. (12) 



2) Case Qi ^ i). By the formula (El we obtain 

U ^rH^i(Ai)), 

Aie7ro-i(Ao) 



where r2i(Ai) is an orbit of Ai G 0* with respect to Ad^^. Appling the induction 
assumption, we obtain 

p =\7T-\u)nn\= k"^(^i(Ai))nQ| = 

AiGTro^^Ao) 

J2 \/|^i(Ai)| • l^^l mult(r",ind(r"i(^i),G)). 

Since |ni(Ai)| = we have 

P = Q mult(r",ind(r"i(^i),G)). (13) 

From the other hand, for any polarization po of Aq the subalgebra 



is a polarization for any Ai G 7rQ"^(Ao). Representation ind(^Ao? ^o? P) is a direct 
sum of one dimensional representations where Ai G 7rQ'^(Ao). Therefore, 
md{t^, G) is a direct sum of representations \n(i{T^^^'^^\ G) where Ai G 7ro"^(Ao). 
We obtain 

M = mult (r", ind(t'^, G)) = ^ mult(r", md{T^'^^'\ G)). (14) 
Substituting dUl) in dE]), we verify P = QM. □ 

Corollary 10. The irreducible representation occurs in decomposition of 
ind(t^, G) if and only if the orbit Q has an nonempty intersection with 7t^^{uj). 
Corollary 11. The irreducible representation occurs in decomposition of 
the restriction of representation on the subgroup H if and only if the orbit 
uj lies in 7r(Q). 

Corollary 12. Let Q, Qi, be coadjoint orbits in g*. Denote by \M\ the 
number of elements in the subset 

M = {(Ai, A2) : Ai G Qi, A2 G Ai + A2 G n}. 

Then 

muit(r",r"i(^r"^)= 



Vl^l • l^il • 1^2! 

Proof. We apply theorem E] to the group G xG, its coadjoint orbit Qi x 
subgroup H = {{g,g) : g ^ G} and its orbit cj = {(A, A) : A G Q}. □ 
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